Skip to content

Optimize scope lifecycle hot path#11079

Open
bm1549 wants to merge 3 commits intomasterfrom
brian.marks/perf-scope-lifecycle
Open

Optimize scope lifecycle hot path#11079
bm1549 wants to merge 3 commits intomasterfrom
brian.marks/perf-scope-lifecycle

Conversation

@bm1549
Copy link
Copy Markdown
Contributor

@bm1549 bm1549 commented Apr 10, 2026 •
edited
Loading

What Does This Do

Reduces overhead in the scope activation/close hot path by:

  • Adding isEmpty() early exits in ContinuableScope.onProperClose() and afterActivated() to skip iteration when no listeners are registered, with results cached in local variables to avoid redundant volatile reads
  • Adding scopeState == Stateful.DEFAULT identity check in beforeActivated() to skip AgentSpan.fromContext() and virtual dispatch when profiling is disabled
  • Caching profilingEnabled and hasDepthLimit boolean flags in ContinuableScopeManager to skip per-activation AgentSpan.fromContext(), instanceof, and scopeStack.depth() calls

Motivation

Scope activation and close happen on every instrumented method entry/exit. In the default configuration (no profiling, no scope listeners, no depth limit), these methods were doing unnecessary work on every call:

  • Iterating empty CopyOnWriteArrayLists (the loop body is never entered, but the iteration overhead remains)
  • Calling AgentSpan.fromContext() and instanceof ProfilerContext in createScopeState() only to return Stateful.DEFAULT
  • Computing scopeStack.depth() to compare against Integer.MAX_VALUE

Why this is faster

CopyOnWriteArrayList.isEmpty() is O(1) — it reads the internal array length without entering the iteration path. The results are cached in local boolean variables since the JIT will be conservative with volatile reads from CopyOnWriteArrayList and won't hoist them across uses. The cached boolean flags (profilingEnabled, hasDepthLimit) turn per-invocation virtual dispatch and method calls into simple branch predictions.

Scope listeners are registered once at startup, not concurrently with scope operations, so the widened isEmpty() race window in onProperClose() is not reachable in practice.

Benchmark results (8 threads, JDK 21, macOS aarch64, Fork=1, Warmup=2, Measurement=3)

Benchmark Baseline (ops/us) Optimized (ops/us) Change
activateAndClose 439.83 488.54 +11.1%
nestedActivateAndClose 43.41 63.14 +45.5%
activateSameSpan 234.30 316.69 +35.2%
activeSpanLookup 2151.11 2751.17 +27.9%

Human readability score: 9/10

All changes follow existing codebase patterns (boolean flag caching, isEmpty guards).

Additional Notes

Also adds ScopeLifecycleBenchmark.java JMH benchmark covering scope activate/close, re-activation, nested scopes, and active span lookup.

tag: no release note
tag: ai generated

Contributor Checklist

  • Format the title according to the contribution guidelines
  • Assign the type: and comp: labels
  • Avoid linking keywords — use solves instead

🤖 Generated with Claude Code

@bm1549 @claude
Optimize scope lifecycle by skipping work when listeners empty and pr…
94cfb4f 
…ofiling disabled

tag: no release note
tag: ai generated

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@bm1549 bm1549 added type: enhancement Enhancements and improvements comp: core Tracer core tag: performance Performance related changes tag: no release notes Changes to exclude from release notes labels Apr 10, 2026
@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Apr 10, 2026 •
edited
Loading

Benchmarks

⚠️ Warning: Baseline build not found for merge-base commit. Comparing against the latest commit on master instead.

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master brian.marks/perf-scope-lifecycle
git_commit_date 1775828702 1775830456
git_commit_sha 7426235 f21e7d4
release_version 1.62.0-SNAPSHOT~742623564b 1.62.0-SNAPSHOT~f21e7d43dc
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1775832355 1775832355
ci_job_id 1585587622 1585587622
ci_pipeline_id 107111829 107111829
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-qaq2d1fb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-qaq2d1fb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 11 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~f21e7d43dc, baseline=1.62.0-SNAPSHOT~742623564b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.061 s) : 0, 1060608
Total [baseline] (8.799 s) : 0, 8798735
Agent [candidate] (1.055 s) : 0, 1054694
Total [candidate] (8.818 s) : 0, 8818099
section iast
Agent [baseline] (1.219 s) : 0, 1219402
Total [baseline] (9.539 s) : 0, 9539204
Agent [candidate] (1.225 s) : 0, 1225316
Total [candidate] (9.568 s) : 0, 9567635
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent iast 1.219 s 158.795 ms (15.0%)
Total tracing 8.799 s -
Total iast 9.539 s 740.469 ms (8.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent iast 1.225 s 170.622 ms (16.2%)
Total tracing 8.818 s -
Total iast 9.568 s 749.536 ms (8.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~f21e7d43dc, baseline=1.62.0-SNAPSHOT~742623564b

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.226 ms) : 0, 1226
crashtracking [candidate] (1.213 ms) : 0, 1213
BytebuddyAgent [baseline] (635.924 ms) : 0, 635924
BytebuddyAgent [candidate] (630.936 ms) : 0, 630936
AgentMeter [baseline] (29.688 ms) : 0, 29688
AgentMeter [candidate] (29.421 ms) : 0, 29421
GlobalTracer [baseline] (249.498 ms) : 0, 249498
GlobalTracer [candidate] (248.587 ms) : 0, 248587
AppSec [baseline] (31.986 ms) : 0, 31986
AppSec [candidate] (32.025 ms) : 0, 32025
Debugger [baseline] (59.166 ms) : 0, 59166
Debugger [candidate] (59.064 ms) : 0, 59064
Remote Config [baseline] (591.177 µs) : 0, 591
Remote Config [candidate] (597.373 µs) : 0, 597
Telemetry [baseline] (8.026 ms) : 0, 8026
Telemetry [candidate] (8.02 ms) : 0, 8020
Flare Poller [baseline] (8.342 ms) : 0, 8342
Flare Poller [candidate] (8.872 ms) : 0, 8872
section iast
crashtracking [baseline] (1.223 ms) : 0, 1223
crashtracking [candidate] (1.217 ms) : 0, 1217
BytebuddyAgent [baseline] (798.31 ms) : 0, 798310
BytebuddyAgent [candidate] (801.109 ms) : 0, 801109
AgentMeter [baseline] (11.328 ms) : 0, 11328
AgentMeter [candidate] (11.417 ms) : 0, 11417
GlobalTracer [baseline] (238.338 ms) : 0, 238338
GlobalTracer [candidate] (240.242 ms) : 0, 240242
IAST [baseline] (25.743 ms) : 0, 25743
IAST [candidate] (26.74 ms) : 0, 26740
AppSec [baseline] (31.617 ms) : 0, 31617
AppSec [candidate] (31.098 ms) : 0, 31098
Debugger [baseline] (59.227 ms) : 0, 59227
Debugger [candidate] (59.646 ms) : 0, 59646
Remote Config [baseline] (524.429 µs) : 0, 524
Remote Config [candidate] (554.939 µs) : 0, 555
Telemetry [baseline] (12.86 ms) : 0, 12860
Telemetry [candidate] (13.19 ms) : 0, 13190
Flare Poller [baseline] (3.443 ms) : 0, 3443
Flare Poller [candidate] (3.497 ms) : 0, 3497
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~f21e7d43dc, baseline=1.62.0-SNAPSHOT~742623564b

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.061 s) : 0, 1061397
Total [baseline] (11.169 s) : 0, 11169072
Agent [candidate] (1.063 s) : 0, 1063299
Total [candidate] (11.15 s) : 0, 11150327
section appsec
Agent [baseline] (1.249 s) : 0, 1248814
Total [baseline] (11.06 s) : 0, 11060276
Agent [candidate] (1.244 s) : 0, 1244102
Total [candidate] (11.077 s) : 0, 11077257
section iast
Agent [baseline] (1.23 s) : 0, 1230441
Total [baseline] (11.163 s) : 0, 11162502
Agent [candidate] (1.222 s) : 0, 1222167
Total [candidate] (11.308 s) : 0, 11307860
section profiling
Agent [baseline] (1.183 s) : 0, 1182792
Total [baseline] (11.131 s) : 0, 11130907
Agent [candidate] (1.183 s) : 0, 1183306
Total [candidate] (11.024 s) : 0, 11023906
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent appsec 1.249 s 187.417 ms (17.7%)
Agent iast 1.23 s 169.043 ms (15.9%)
Agent profiling 1.183 s 121.395 ms (11.4%)
Total tracing 11.169 s -
Total appsec 11.06 s -108.797 ms (-1.0%)
Total iast 11.163 s -6.57 ms (-0.1%)
Total profiling 11.131 s -38.166 ms (-0.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.244 s 180.803 ms (17.0%)
Agent iast 1.222 s 158.868 ms (14.9%)
Agent profiling 1.183 s 120.007 ms (11.3%)
Total tracing 11.15 s -
Total appsec 11.077 s -73.07 ms (-0.7%)
Total iast 11.308 s 157.533 ms (1.4%)
Total profiling 11.024 s -126.421 ms (-1.1%) 
gantt
    title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~f21e7d43dc, baseline=1.62.0-SNAPSHOT~742623564b

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.241 ms) : 0, 1241
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (636.356 ms) : 0, 636356
BytebuddyAgent [candidate] (635.762 ms) : 0, 635762
AgentMeter [baseline] (29.669 ms) : 0, 29669
AgentMeter [candidate] (29.54 ms) : 0, 29540
GlobalTracer [baseline] (250.785 ms) : 0, 250785
GlobalTracer [candidate] (249.611 ms) : 0, 249611
AppSec [baseline] (32.33 ms) : 0, 32330
AppSec [candidate] (32.138 ms) : 0, 32138
Debugger [baseline] (60.302 ms) : 0, 60302
Debugger [candidate] (60.235 ms) : 0, 60235
Remote Config [baseline] (598.322 µs) : 0, 598
Remote Config [candidate] (613.537 µs) : 0, 614
Telemetry [baseline] (8.187 ms) : 0, 8187
Telemetry [candidate] (8.119 ms) : 0, 8119
Flare Poller [baseline] (5.774 ms) : 0, 5774
Flare Poller [candidate] (9.839 ms) : 0, 9839
section appsec
crashtracking [baseline] (1.22 ms) : 0, 1220
crashtracking [candidate] (1.209 ms) : 0, 1209
BytebuddyAgent [baseline] (662.279 ms) : 0, 662279
BytebuddyAgent [candidate] (659.714 ms) : 0, 659714
AgentMeter [baseline] (12.096 ms) : 0, 12096
AgentMeter [candidate] (11.985 ms) : 0, 11985
GlobalTracer [baseline] (249.081 ms) : 0, 249081
GlobalTracer [candidate] (248.311 ms) : 0, 248311
IAST [baseline] (24.599 ms) : 0, 24599
IAST [candidate] (24.511 ms) : 0, 24511
AppSec [baseline] (184.492 ms) : 0, 184492
AppSec [candidate] (184.043 ms) : 0, 184043
Debugger [baseline] (65.847 ms) : 0, 65847
Debugger [candidate] (65.409 ms) : 0, 65409
Remote Config [baseline] (600.86 µs) : 0, 601
Remote Config [candidate] (594.881 µs) : 0, 595
Telemetry [baseline] (8.581 ms) : 0, 8581
Telemetry [candidate] (8.543 ms) : 0, 8543
Flare Poller [baseline] (3.55 ms) : 0, 3550
Flare Poller [candidate] (3.562 ms) : 0, 3562
section iast
crashtracking [baseline] (1.246 ms) : 0, 1246
crashtracking [candidate] (1.206 ms) : 0, 1206
BytebuddyAgent [baseline] (807.007 ms) : 0, 807007
BytebuddyAgent [candidate] (800.374 ms) : 0, 800374
AgentMeter [baseline] (11.446 ms) : 0, 11446
AgentMeter [candidate] (11.365 ms) : 0, 11365
GlobalTracer [baseline] (239.346 ms) : 0, 239346
GlobalTracer [candidate] (238.684 ms) : 0, 238684
IAST [baseline] (25.756 ms) : 0, 25756
IAST [candidate] (26.562 ms) : 0, 26562
AppSec [baseline] (32.508 ms) : 0, 32508
AppSec [candidate] (32.549 ms) : 0, 32549
Debugger [baseline] (60.534 ms) : 0, 60534
Debugger [candidate] (57.386 ms) : 0, 57386
Remote Config [baseline] (529.017 µs) : 0, 529
Remote Config [candidate] (517.148 µs) : 0, 517
Telemetry [baseline] (12.081 ms) : 0, 12081
Telemetry [candidate] (13.534 ms) : 0, 13534
Flare Poller [baseline] (3.456 ms) : 0, 3456
Flare Poller [candidate] (3.541 ms) : 0, 3541
section profiling
crashtracking [baseline] (1.182 ms) : 0, 1182
crashtracking [candidate] (1.181 ms) : 0, 1181
BytebuddyAgent [baseline] (690.398 ms) : 0, 690398
BytebuddyAgent [candidate] (690.825 ms) : 0, 690825
AgentMeter [baseline] (9.13 ms) : 0, 9130
AgentMeter [candidate] (9.127 ms) : 0, 9126
GlobalTracer [baseline] (207.27 ms) : 0, 207270
GlobalTracer [candidate] (207.091 ms) : 0, 207091
AppSec [baseline] (32.445 ms) : 0, 32445
AppSec [candidate] (32.528 ms) : 0, 32528
Debugger [baseline] (65.658 ms) : 0, 65658
Debugger [candidate] (65.278 ms) : 0, 65278
Remote Config [baseline] (574.352 µs) : 0, 574
Remote Config [candidate] (568.519 µs) : 0, 569
Telemetry [baseline] (7.805 ms) : 0, 7805
Telemetry [candidate] (7.877 ms) : 0, 7877
Flare Poller [baseline] (3.525 ms) : 0, 3525
Flare Poller [candidate] (3.564 ms) : 0, 3564
ProfilingAgent [baseline] (93.567 ms) : 0, 93567
ProfilingAgent [candidate] (93.969 ms) : 0, 93969
Profiling [baseline] (94.128 ms) : 0, 94128
Profiling [candidate] (94.545 ms) : 0, 94545
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master brian.marks/perf-scope-lifecycle
git_commit_date 1775828702 1775830456
git_commit_sha 7426235 f21e7d4
release_version 1.62.0-SNAPSHOT~742623564b 1.62.0-SNAPSHOT~f21e7d43dc
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1775832764 1775832764
ci_job_id 1585587626 1585587626
ci_pipeline_id 107111829 107111829
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-fr2abxmf 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-fr2abxmf 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 19 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~f21e7d43dc, baseline=1.62.0-SNAPSHOT~742623564b
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.47 ms) : 19270, 19670
.   : milestone, 19470,
appsec (18.704 ms) : 18517, 18891
.   : milestone, 18704,
code_origins (17.578 ms) : 17406, 17749
.   : milestone, 17578,
iast (17.802 ms) : 17627, 17977
.   : milestone, 17802,
profiling (18.237 ms) : 18055, 18419
.   : milestone, 18237,
tracing (17.945 ms) : 17767, 18123
.   : milestone, 17945,
section candidate
no_agent (18.348 ms) : 18158, 18538
.   : milestone, 18348,
appsec (18.617 ms) : 18430, 18804
.   : milestone, 18617,
code_origins (17.74 ms) : 17568, 17913
.   : milestone, 17740,
iast (17.705 ms) : 17529, 17880
.   : milestone, 17705,
profiling (18.208 ms) : 18024, 18392
.   : milestone, 18208,
tracing (17.858 ms) : 17684, 18032
.   : milestone, 17858,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.47 ms [19.27 ms, 19.67 ms] -
appsec 18.704 ms [18.517 ms, 18.891 ms] -766.1 µs (-3.9%)
code_origins 17.578 ms [17.406 ms, 17.749 ms] -1.892 ms (-9.7%)
iast 17.802 ms [17.627 ms, 17.977 ms] -1.668 ms (-8.6%)
profiling 18.237 ms [18.055 ms, 18.419 ms] -1.233 ms (-6.3%)
tracing 17.945 ms [17.767 ms, 18.123 ms] -1.525 ms (-7.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.348 ms [18.158 ms, 18.538 ms] -
appsec 18.617 ms [18.43 ms, 18.804 ms] 268.837 µs (1.5%)
code_origins 17.74 ms [17.568 ms, 17.913 ms] -607.71 µs (-3.3%)
iast 17.705 ms [17.529 ms, 17.88 ms] -643.436 µs (-3.5%)
profiling 18.208 ms [18.024 ms, 18.392 ms] -140.254 µs (-0.8%)
tracing 17.858 ms [17.684 ms, 18.032 ms] -490.094 µs (-2.7%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~f21e7d43dc, baseline=1.62.0-SNAPSHOT~742623564b
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.222 ms) : 1211, 1233
.   : milestone, 1222,
iast (3.177 ms) : 3135, 3219
.   : milestone, 3177,
iast_FULL (5.761 ms) : 5703, 5818
.   : milestone, 5761,
iast_GLOBAL (3.73 ms) : 3657, 3802
.   : milestone, 3730,
profiling (2.129 ms) : 2110, 2149
.   : milestone, 2129,
tracing (1.893 ms) : 1877, 1909
.   : milestone, 1893,
section candidate
no_agent (1.237 ms) : 1225, 1249
.   : milestone, 1237,
iast (3.287 ms) : 3240, 3334
.   : milestone, 3287,
iast_FULL (5.894 ms) : 5835, 5953
.   : milestone, 5894,
iast_GLOBAL (3.628 ms) : 3565, 3690
.   : milestone, 3628,
profiling (2.143 ms) : 2124, 2162
.   : milestone, 2143,
tracing (1.939 ms) : 1922, 1957
.   : milestone, 1939,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.222 ms [1.211 ms, 1.233 ms] -
iast 3.177 ms [3.135 ms, 3.219 ms] 1.955 ms (160.0%)
iast_FULL 5.761 ms [5.703 ms, 5.818 ms] 4.539 ms (371.4%)
iast_GLOBAL 3.73 ms [3.657 ms, 3.802 ms] 2.508 ms (205.2%)
profiling 2.129 ms [2.11 ms, 2.149 ms] 907.578 µs (74.3%)
tracing 1.893 ms [1.877 ms, 1.909 ms] 670.732 µs (54.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.237 ms [1.225 ms, 1.249 ms] -
iast 3.287 ms [3.24 ms, 3.334 ms] 2.05 ms (165.7%)
iast_FULL 5.894 ms [5.835 ms, 5.953 ms] 4.657 ms (376.4%)
iast_GLOBAL 3.628 ms [3.565 ms, 3.69 ms] 2.39 ms (193.2%)
profiling 2.143 ms [2.124 ms, 2.162 ms] 905.948 µs (73.2%)
tracing 1.939 ms [1.922 ms, 1.957 ms] 702.198 µs (56.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master brian.marks/perf-scope-lifecycle
git_commit_date 1775828702 1775830456
git_commit_sha 7426235 f21e7d4
release_version 1.62.0-SNAPSHOT~742623564b 1.62.0-SNAPSHOT~f21e7d43dc
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1775832579 1775832579
ci_job_id 1585587628 1585587628
ci_pipeline_id 107111829 107111829
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-i7vt0jw7 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-i7vt0jw7 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~f21e7d43dc, baseline=1.62.0-SNAPSHOT~742623564b
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.99 s) : 14990000, 14990000
.   : milestone, 14990000,
appsec (14.731 s) : 14731000, 14731000
.   : milestone, 14731000,
iast (18.404 s) : 18404000, 18404000
.   : milestone, 18404000,
iast_GLOBAL (18.19 s) : 18190000, 18190000
.   : milestone, 18190000,
profiling (14.972 s) : 14972000, 14972000
.   : milestone, 14972000,
tracing (15.034 s) : 15034000, 15034000
.   : milestone, 15034000,
section candidate
no_agent (15.752 s) : 15752000, 15752000
.   : milestone, 15752000,
appsec (14.828 s) : 14828000, 14828000
.   : milestone, 14828000,
iast (18.291 s) : 18291000, 18291000
.   : milestone, 18291000,
iast_GLOBAL (18.181 s) : 18181000, 18181000
.   : milestone, 18181000,
profiling (15.074 s) : 15074000, 15074000
.   : milestone, 15074000,
tracing (14.886 s) : 14886000, 14886000
.   : milestone, 14886000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.99 s [14.99 s, 14.99 s] -
appsec 14.731 s [14.731 s, 14.731 s] -259.0 ms (-1.7%)
iast 18.404 s [18.404 s, 18.404 s] 3.414 s (22.8%)
iast_GLOBAL 18.19 s [18.19 s, 18.19 s] 3.2 s (21.3%)
profiling 14.972 s [14.972 s, 14.972 s] -18.0 ms (-0.1%)
tracing 15.034 s [15.034 s, 15.034 s] 44.0 ms (0.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.752 s [15.752 s, 15.752 s] -
appsec 14.828 s [14.828 s, 14.828 s] -924.0 ms (-5.9%)
iast 18.291 s [18.291 s, 18.291 s] 2.539 s (16.1%)
iast_GLOBAL 18.181 s [18.181 s, 18.181 s] 2.429 s (15.4%)
profiling 15.074 s [15.074 s, 15.074 s] -678.0 ms (-4.3%)
tracing 14.886 s [14.886 s, 14.886 s] -866.0 ms (-5.5%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~f21e7d43dc, baseline=1.62.0-SNAPSHOT~742623564b
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.483 ms) : 1471, 1494
.   : milestone, 1483,
appsec (2.536 ms) : 2481, 2591
.   : milestone, 2536,
iast (2.27 ms) : 2200, 2340
.   : milestone, 2270,
iast_GLOBAL (2.314 ms) : 2244, 2384
.   : milestone, 2314,
profiling (2.098 ms) : 2043, 2153
.   : milestone, 2098,
tracing (2.082 ms) : 2028, 2136
.   : milestone, 2082,
section candidate
no_agent (1.481 ms) : 1470, 1493
.   : milestone, 1481,
appsec (3.808 ms) : 3584, 4032
.   : milestone, 3808,
iast (2.268 ms) : 2198, 2337
.   : milestone, 2268,
iast_GLOBAL (2.312 ms) : 2242, 2382
.   : milestone, 2312,
profiling (2.094 ms) : 2039, 2150
.   : milestone, 2094,
tracing (2.075 ms) : 2021, 2128
.   : milestone, 2075,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.483 ms [1.471 ms, 1.494 ms] -
appsec 2.536 ms [2.481 ms, 2.591 ms] 1.053 ms (71.0%)
iast 2.27 ms [2.2 ms, 2.34 ms] 787.232 µs (53.1%)
iast_GLOBAL 2.314 ms [2.244 ms, 2.384 ms] 831.646 µs (56.1%)
profiling 2.098 ms [2.043 ms, 2.153 ms] 615.514 µs (41.5%)
tracing 2.082 ms [2.028 ms, 2.136 ms] 599.247 µs (40.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.481 ms [1.47 ms, 1.493 ms] -
appsec 3.808 ms [3.584 ms, 4.032 ms] 2.327 ms (157.1%)
iast 2.268 ms [2.198 ms, 2.337 ms] 786.465 µs (53.1%)
iast_GLOBAL 2.312 ms [2.242 ms, 2.382 ms] 830.715 µs (56.1%)
profiling 2.094 ms [2.039 ms, 2.15 ms] 613.257 µs (41.4%)
tracing 2.075 ms [2.021 ms, 2.128 ms] 593.331 µs (40.1%)

@bm1549 @claude
Fix activeSpanLookup benchmark to exercise live scope stack
1e13d53 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@bm1549 bm1549 marked this pull request as ready for review April 10, 2026 13:30
@bm1549 bm1549 requested review from a team as code owners April 10, 2026 13:30
@bm1549 bm1549 requested review from mhlidd and removed request for a team April 10, 2026 13:30
@github-actions github-actions bot added the tag: ai generated Largely based on code generated by an AI or LLM label Apr 10, 2026
@dougqh dougqh requested a review from mcculls April 10, 2026 13:43
dougqh
dougqh reviewed Apr 10, 2026
View reviewed changes
dd-trace-core/src/main/java/datadog/trace/core/scopemanager/ContinuableScope.java Outdated
listener.afterScopeClosed();
} catch (Exception e) {
ContinuableScopeManager.log.debug("ScopeListener threw exception in close()", e);
if (!scopeManager.scopeListeners.isEmpty()) {
Copy link
Copy Markdown
Contributor

@dougqh dougqh Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change makes sense. Although in my experience usually the Iterator gets stack allocated by the JIT, so I'm not sure the provided explanation makes sense here.

That said, we'd have to look at an allocation profile to know for certain.

Copy link
Copy Markdown
Contributor Author

@bm1549 bm1549 Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dug into this and the more accurate explanation is

It avoids the iteration dispatch path entirely, whether or not the JIT scalar-replaces the iterator, the isEmpty() check is fewer instructions and method calls than entering and immediately exiting the for-each loop.

dougqh
dougqh requested changes Apr 10, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@dougqh dougqh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor suggestions on further improvements

I'd also like Stuart to review, since he's more familiar with this code

@bm1549 @claude
Cache isEmpty() results in local variables to avoid redundant volatil…
f21e7d4 
…e reads

Addresses review feedback from dougqh: the JIT will be conservative
with volatile reads from CopyOnWriteArrayList.isEmpty() and won't
hoist them, so we cache in locals ourselves.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@dougqh
Copy link
Copy Markdown
Contributor

dougqh commented Apr 10, 2026

In Spring Petclinic throughput tests...

Prior to the isEmpty being pulled into a local variable, this change was negative by 2-5% / 500-1000 requests/sec.
Presumably from memory access, since the CopyOnWriteArrayList must be checked each time.
With a non-concurrent List, the JIT would have been able to do this optimization for us.

After pulling isEmpty into a local variable, this change is mostly neutral throughput wise.
That's in line my expectations given that scopes only account for < 1% of CPU time.
And the Iterator construction while theoretically a problem is usually optimized away by the JIT.

Overall, this change seems good to me. Unfortunately, we just don't have a lot of easy big wins anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dougqh dougqh dougqh approved these changes

@mhlidd mhlidd Awaiting requested review from mhlidd mhlidd is a code owner automatically assigned from DataDog/apm-sdk-capabilities-java

@mcculls mcculls Awaiting requested review from mcculls

Assignees

No one assigned

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes tag: performance Performance related changes type: enhancement Enhancements and improvements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@bm1549 @dougqh